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STUDENT ACHIEVEMENT TESTING PROGRAM 


Preface 


Following each administration of the student achievement tests, a report of 
provincial results is prepared. This document is the report for the June 1987 
administration. Each school and school jurisdiction also receives a report of 
test results for its students. 


The Provincial Report consists of general information about the student 
achievement testing program, followed by information specific to each subject. 


The Achievement Testing Program provides information significant at the 
provincial and local levels about student knowledge, understanding, and skills 
in relation to program objectives. 


The achievement tests are specific to the program of studies prescribed by the 
Minister of Education. Curriculum specifications for each subject area, 
provided by the Curriculum Design Branch and the Language Services Branch of 
Alberta Education, identify the major content areas, the specific learning 
objectives within each area, and the emphasis that each objective is to, ; oS mM 


receive. The test questions reflect these curriculum specifications. “ee Cae. a 
“wm Sea 

The achievement tests, administered on a cyclical basis, are in four subyeCh 5 an Soc 

areas: language arts, social studies, mathematics, and science; and at three. ie m 

grade levels: 3, 6, and 9. In 1987, achievement tests were administered in” 4. 

Grade 3 Science, Grade 6 Mathematics, and Grade 9 Social Studies. gu see 
jnarawiig 7 S Og SAU : ‘Im cee LA 

Following the achievement test administration in June of each year, the - 


results are reported to each school jurisdiction. These district profiles 
include results for each school and each student. Individual statements of 
results are not issued to students. 


This Provincial Report .is designed to assist school administrators and 
teachers in interpreting their achievement test results. 


Exemptions from the Achievement Testing Program 


Under normal circumstances, the following students are exempt from achievement 
testing: 


- Students who participated in Special Education programs 


- Students who were taught the subject being tested in another semester or 
year 


- Students enrolled in an English as a Second Language program 


- Students who were taught the specific subject being tested in a language 
other than English 


-— Students who received exemption as approved by the Director of the Student 
Evaluation and Records Branch 
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GRADE 3 SCIENCE ACHIEVEMENT TEST 


General Description 


The Grade 3 Science Achievement Test consists of 50 multiple-choice questions 
which are answered in the booklet in machine~scorable format. The time 
allotted for writing the test is 50 minutes. The test is divided into two 
parts of equal length in recognition of the need to provide Grade 3 students 
with a break. Therefore, each part has 25 multiple-choice questions with a 
time allocation of 25 minutes. 


Content 


The test is based on the Grade 3 Science Curriculum Specifications (Revised, 
August 1986). The four components of the Grade 3 curriculum are: 


Components Emphasis (%) 
Process Skills 55 
Subject Matter 20 
Psychomotor Skills 15 
Attitudes 10 


Test questions are drawn from the process skills and subject matter 
components. The psychomotor skills and attitudes components should be 
reflected in the teacher's evaluation of the student. 


The emphasis given to the process skills and subject matter components in the 
Grade 3 Achievement Test is shown in the circle graph below: 
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The subject matter component includes questions from each theme in the 
Division I Science core. These themes are listed below. 


1. Matter and Energy 
2. Living Things and the Environment 
3. Earth, Space, and Time 


The emphasis given to each theme and to the process skills is shown in the 
circle graph below. 
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MATTER and ENERGY 


LIVING THINGS and ENVIRONMENT EARTH, SPACE and TIME 


The test questions represent the cognitive levels of knowledge, application, 
and understanding. 


Approximately 20% of the questions test knowledge of subject matter. 
Approximately 80% of the questions test application and understanding. 

It is assumed that students are familiar with the terminology used in the 
curriculum specifications (e.g. observing, predicting, measuring, classifying, 


mass, food chain, etc). 


The Grade 3 Science Achievement Test is based on the Division I Curriculum 
Specifications and includes material covered in grades 1, 2, and 3. 


Blueprint of the Achievement Test 


The blueprint for the Grade 3 Science Achievement Test is presented in Table 1. 


Table 1: Grade 3 Science -— Test Blueprint: 
Question Numbers and Per Cent Emphasis 


Content 


Knowledge 


Subject Matter 
Matter and Energy 45 


Living Things and the 
Environment 


Process Skills 2, 3, 4, 5, 6, 
8, 9, 10, 12, 
13, 14, 15, 17, 
18, 26, 28, 29, 
32, 33, 34, 35, 
36, 37, 38, 39, 
40, 41, 42, 43 


phasis (%) 


Absentees and Exemptions 


Table 2: Grade 3 Science - Students Tested, Absentees, and Exemptions 


Number of 
Students Per Cent 
Students tested 31 384 89.4 
Students absent - 1 067 3.0 
Students exempt: 
Special Needs 676 1.9 
Not taught subject this term 536 1.5 
ESL Classes 273 0.8 
Language of Instruction not 
English 1 152 3.3 
Other (Approved by Director) 37 0.1 
Total Exempt 2 674 7.6 


Note: the data are incomplete as not all schools returned absentee and 
exemption reports. 
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RESULTS Tut: 


The report is based on the results for 31 384 students from public, separate, 


and private schools. 


Total Test 


Since over 30 800 of the students completed the test, it was concluded that 
sufficient time was allowed to write the test. Test statistics in percentage 
points are as follows: 


Provincial Average - 71.3 
Standard Deviation - 15.9 


The KR-20 reliability coefficient for the test was 0.86. 

Standards for the Minimum Acceptable and Outstanding Performance Levels 

The minimum acceptable and outstanding performance levels of achievement were 
established by a standard setting procedure*., For the Grade 3 Science 


Achievement Test the standards established were as follows: 


To meet the minimum acceptable performance level, students must achieve a 
raw score of 24. 


To meet the outstanding performance level, students must achieve a raw 
score of 44, 


Based on these standards the results reveal that: 


91.2 per cent of students performed at or above the minimum acceptable 
performance level. 


15.8 per cent of students performed at or above the outstanding 
performance level. 


Reporting Categories 


The questions have been grouped according to reporting categories. Provincial 
averages for these categories and for the total test were computed and rounded 
to one decimal. Consequently, the sum of the averages is not exactly the same 
as the average for the total test. Provincial averages for these reporting 
categories and the total test are presented in Table 3 on page 6. 


*An explanation of the standard setting procedure is given in Appendix A. 
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Table 3: Grade 3 Science - Provincial Averages for Total Test, 
Process Skills, and Subject Matter 


Number of Average Standard Deviation 

Reporting Category Questions (Raw Score) (Raw Score) 
Total Test 50 35.6 8.0 
Subject Matter 

Matter and Energy 6 4.0 1.4 

Living Things and the 

Environment 6 4.6 1.3 

Earth, Space, and Time 2 1.5 0.7 
Process Skills 36 25.6 5.9 
Cognitive Level 

Knowledge 10 7.0 2.0 

Application 40 28.7 6.4 


Frequency Distribution of Raw Scores 


Table 4 presents the relative frequency and the cumulative frequency for each 
raw score. The range of scores was from 0 to 50. Any relative frequency or 
any cumulative frequency smaller than 0.05 was rounded to 0.0. For example, 
5 students or 0.02% obtained a score of 6 but the relative frequency for this 
score is recorded as 0.0. 


Table 4: Grade 3 Science — Frequency Distribution of Raw Scores 


Raw Relative Cumulative Raw Relative Cumulative 
Score Frequency* Frequency** Score Frequency* Frequency** 
(%) (%) (%) (%) 
0 0.0 0.0 26 2.0 13.8 
1 0.0 0.0 27 2.0 15.8 
2 0.0 0.0 28 2.5 18.3 
3 0.0 0.0 29 2.6 20.9 
4 0.0 0.0 30 2.9 23.7 
5 0.0 0.0 31 3.1 26.8 
6 0.0 0.0 32 3.5 30.3 
7 0.0 0.1 33 3.8 34.1 
8 0.1 0.1 34 4.3 38.5 
] 0.0 0.2 35 4.7 43.2 
10 0.1 0.3 36 4.7 47.9 
11 0.2 0.4 37 5.0 52.8 
12 0.2 0.7 38 5.1 58.0 
13 0.3 1.0 39 5.6 63.5 
14 0.4 1.4 40 5.3 68.9 
15 0.4 1.8 41 5.5 74.4 
16 0.6 2.4 42 5.1 79.4 
17 0.6 2.9 43 4.8 84.2 
18 0.7 3.6 44 4.4 88.6 
19 0.7 4.3 45 3.7 92.3 
20 0.9 5.2 46 3.1 95.3 
21 1.0 6.2 47 2.3 97.7 
22 1.2 7.4 48 1.4 99.0 
23 1.4 8.8 49 0.8 99.8 
24 1.4 10.2 50 0.2 100.0 
25 1.6 11.8 


*Relative frequency: the percentage of students who obtained each score. 


**Cumulative frequency: the percentage of students who scored at or below each 
score. 


i: -7- 


Results for Individual Questions 


The percentage of students choosing each response is given in Table 5. The 
correct response (key) for each question is also identified. 


Table 5: Grade 3 Science - Results for Individual Questions 


Distribution of Distribution of 
Responses* (%) Responses* (%) 
Item | Key 
A B Cc D 
B 3 ol 26 D 6.2 5.8 5.9 81.6 
A 9 -6 27 D 4.3 14.0 3.2 77.2 
D 4 5 28 Cc 41.2 6.5 47.4 4.3 
Cc 8 2 29 Cc 13.6 9.7 67.2 8.1 
D 8 .7 30 B 2.0 60.9 13.5 22.7 
A 4 6 31 A 80.2 6.8 4.9 6.9 
D “a 3 9 32 Cc 5.7 4.1 87.0 2.2 
B ol 3 9 33 A 65.5 14.7 16.2 2.2 
A 6 -6 .2 34 A 84.8 5.0 3.0 5.4 
D 5 6 el 35 B 19.0 70.5 3.5 5.8 
D 6 9 4 36 B 2.2 91.7 2.1 2.1 
Cc 5 3 ol 37 D 5.7 10.3 13.5 69.1 
A .7 .7 .0 38 B 10.7 45.0 8.0 35.6 
D 6 .8 .2 39 Cc 10.1 12.9 67.4 8.5 
B 6 4 -6 40 B 2.4 94.6 1.4 0.6 
A 4 6 9 41 Cc 17.1 9.0 63.9 8.3 
Cc .0 .6 2 42 Cc 6.2 14.1 71.8 6.6 
Cc .7 3 .8 43 A 76.5 5.3 3.7 8.4 
B ‘5 3 -0 44 B 14.7 59.0 10.7 11.5 
Cc el 2 .0 45 A 68.1 7.7 5.4 15.6 
B 1 1 .0 46 D 3.7 6.8 15.4 73.1 
B 9 .7 5 47 A 93.8 1.7 1.9 1.5 
D .9 ol 8 48 Cc 2.9 6.0 87.8 2.0 
Cc .0 ol .6 49 B 18.7 57.9 9.2 12.3 
A 4 .0 -0 50 D 6.6 8.8 14.5 68.4 


*The sum of the percentages for each question may be less than 100% because the 
No Response category is not included. The No Response category does not ae 
exceed 6.0% for any one of these questions. 


**Due to a technical problem, the distribution of responses for students 
choosing alternatives C and D are unavailable for Question 6. 
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Distribution of Jurisdiction Levels of Achievement 


Table 6 indicates the percentage of jurisdictions classified as significantly 
above, not different from, or below the provincial average for each subtest. 


Table 6: Grade 3 Science - Distribution of Jurisdiction 
Levels of Achievement 


Distribution of Jurisdiction Levels 
of Achievement 


Reporting Category Below Not Different Above 


Provincial from Provincial Provincial 
Average Average Average 
(%) (%) (%) 
Total Test 16.3 52.6 31.2 
Subject Matter 
Matter and Energy 17.1 56.2 26.7 
Living Things and the 
Environment 10.4 62.6 27.0 
Earth, Space, and Time 11.2 73.7 15.1 
Process Skills 17.3 54.2 28.5 
Cognitive Level 
Knowledge 17.8 58.4 23.8 
Application 16.0 53.1 31.0 


GRADE 6 MATHEMATICS ACHIEVEMENT TEST 


General Description 


The Grade 6 Mathematics Achievement Test consists of two sections. Section 1 
contains 55 questions covering numeration, operations and properties, 
measurement, geometry, graphing, and problem-solving strategies. Section 2 
comprises five basic-fact tests in addition, subtraction, multiplication, 
division, and mixed operations. Each basic-fact test contains 48 questions. 


The total time allotted for writing the test is 70 minutes: 60 minutes for 
Section 1, and two minutes for each basic-fact test in Section 2. 


All questions are multiple-choice with four alternatives. Students answer 
questions on machine-scorable answer sheets for Section 1. For the basic~fact 
tests in Section 2, students answer questions in the test booklets by filling 
in a small circle beside the correct response. There was no separate answer 
sheet for basic-fact tests. 


Content 


The test is based on the Grade 6 Mathematics Curriculum Specifications 
(revised May, 1986). The four components of the Grade 6 curriculum are: 


Component Emphasis (%) 
Subject Matter 60 
Problem-Solving Skills 20 
Psychomotor Skills 10 
Attitude 10 


Test questions are drawn only from the subject matter and problem-solving 
skills components. The psychomotor skills and attitude components should be 
reflected in the teacher's evaluation of the student. 
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Section 1: Subject Matter and Problem-Solving Skills 


The emphasis given to the subject matter and problem-solving skills components 
is shown in the circle graph below. 


Problem- 

Solving 
Skills 
20.0% 


The subject matter component covers five strands: numeration, operations and 
properties, measurement, geometry, and graphing. The problem-solving skills 
component is divided into two parts: problem-solving integrated with subject 
matter and problem-solving strategies. The emphasis given to each subject 
matter strand and to each part of the problem-solving skills component is 
shown in the circle graph below. 
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Questions on subject matter measure student achievement at three cognitive 
levels: knowledge, comprehension, and application. 


Blueprint for the Achievement Test 


The question numbers and the per cent emphasis are presented in Table 7. 


Table 7: Grade 6 Mathematics - Test Blueprint for Section 1: 
Question Numbers and Per Cent Emphasis 


Subject Matter by Cognitive Levels 


Problem 
Solving 


Knowledge | Comprehension | Application 


Numeration 


Operations and 15,16,19,20, 17,21,25, 
Properties 23,24,26,27, 30 
28 


Measurement 34,35,37 


Geometry 


49 ,50,53 


Problem-Solving 
Strategies 


Section I 
Emphasis 
(%) 
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Section 2: Basic Facts 


This section of the test consists of five timed tests. They measure student 
mastery of basic facts involving sums and minuends to 18, and products and 
dividends to 81. 


The order of the basic-fact tests, the number of questions on each test, and 
the time allotted to write each test are presented in Table 8 below. 


Table 8: Grade 6 Mathematics - 
Test Blueprint for Section 2: Basic Facts 


Operations No. of Questions 


Addition 
Subtraction 
Multiplication 
Division 

Mixed Operations 


Absentees and Exemptions 


Table 9: Grade 6 Mathematics - Students Tested, 
Absentees, and Exemptions 


Number of 
Students Per Cent 


Students tested 31 130 
Students absent 1 074 
Students exempt: 


Special Needs 585 
Not taught subject this term 8 
ESL Classes 115 
Language of Instruction not 

English : 2 
Other (Approved by Director) 71 
Total Exempt 781 


Note: the data are incomplete as not all schools returned absentee and 
exemption reports. 
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RESULTS 


The report is based on the results for 31 130 students from public, separate, 
and private schools. Results for Section 1 and Section 2 are reported 
separately. 


Results for Section 1 - Subject Matter and Problem-Solving Skills 

Since over 30 500 of the students completed the test, it was concluded that 
sufficient time was allowed to write Section 1. Test statistics in percentage 
points are as follows: 


Provincial Average - 69.7 
Standard Deviation - 17.3 


The KR-20 reliability coefficient for the Section 1 was 0.90. 


Standards for Minimum Acceptable and Outstanding Performance Levels 

The minimum acceptable and outstanding performance levels were established 
using a standard setting procedure*. For Section 1 of the Grade 6 Mathematics 
Achievement Test, the standards established were as follows: 


To meet the minimum acceptable performance level, students must achieve a 
raw score of 24. 


To meet the outstanding performance level, students must achieve a raw 
score of 50. 


Based on these standards, results for Section 1 of the Grade 6 Mathematics 
Achievement Test revealed that: 


92.1 per cent of students performed at or above the minimum acceptable 
performance level. 


12.3 per cent of students performed at or above the outstanding 
performance level. 


*An explanation of the standard setting procedure is given in Appendix A. 
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Results for Section 1, Subject Strands, Cognitive Levels, and Problem Solving 


The questions have been grouped according to reporting categories. Provincial 
averages for each reporting category and for the total of Section 1 were 
computed and rounded to one decimal. Consequently, the sum of the reporting 
! category averages is not exactly the same as the average for Section 1 (total). 
, Provincial averages for these reporting categories and for Section 1 (total) 
are presented in Table 10. 


Table 10: Grade 6 Mathematics - Provincial Averages for Section 1 (total), 
Subject Strands, Cognitive Levels, and Problem Solving 
Number of Average Standard Deviation 
Reporting Category Questions Raw Score (Raw Score) 
Section 1 (total) 55 38.4 9.5 
Subject Strand 
Numeration 13 9.0 2.7 
Operations and Properties 16 11.3 3.2 
Measurement 11 7.5 2.2 
Geometry 6 4.1 1.4 
Graphing 6 ~ 4.4 1.5 
Problem-Solving Strategies 3 2.1 0.8 
A Cognitive Level 
Knowledge 13 9.5 2.4 
Comprehension 18 12.7 3.5 
Application 13 9.2 2.8 
Problem-Solving Skills 11 7.0 2.4 


245.5 


Section 1: Frequency Distribution of Raw Scores 


Table 11 presents total raw scores for Section 1 as well as the relative 
frequency and the cumulative frequency for each raw score. The range of 
scores was from 0 to 55. Any relative frequency or any cumulative frequency 
smaller than 0.05 was rounded to 0.0. For example, 13 students or 0.04% 
obtained a score of 8, but the relative frequency for this score is recorded 
as 0.0. 


Table 11: Grade 6 Mathematics: Section 1 
Frequency Distribution of Raw Scores 


Relative Cumulative Relative Cumulative 
Frequency* Frequency** Raw Frequency* Frequency** 
Score (%) (%) Score (%) (%) 
0 0.0 0.0 293 2.4 19.0 
1 0.0 0.0 30 2.5 21.5 
2 0.0 0.0 31 2.6 24.1 
3 0.0 0.0 32 2.9 27.1 
4 0.0 0.0 33 2.8 29.9 
5 0.0 0.0 34 3.2 33.1 
6 0.0 0.0 35 3.3 36.4 
7 0.0 0.0 36 3.2 39.6 
8 0.0 0.1 37 3.4 43.1 
9 0.0 0.1 38 3.7 46.8 
10 0.1 0.2 39 3.6 50.4 
11 0.1 0.3 40 3.9 54.3 
12 0.2 0.5 41 3.8 58.1 
13 0.2 0.7 42 4.0 62.1 
14 0.2 0.9 43 3.8 65.9 
15 0.3 1.2 44 3.8 69.7 
16 0.4 1.6 45 3.9 73.6 
17 0.5 2.1 46 3.8 77.4 
18 0.7 2.8 47 3.5 80.9 
19 0.7 3.4 48 3.3 84.2 
20 0.9 4.4 49 3.5 87.7 
21 1.1 5.4 50 3.0 90.7 
22 1.2 6.6 51 2.9 93.6 
23 1.3 7.9 52 2.4 96.1 
24 1.4 9.3 53 2.0 98.1 
25 1.5 10.9 54 1.3 99.4 
26 1.7 12.5 55 0.6 100.00 
27 2.0 14.5 
28 2.1 16.6 


*Relative frequency: the percentage of students who obtained each score. 


**Cumulative frequency: the percentage of students who scored at or below each 
score. 
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Section 1: Results for Individual Questions 


The percentage of students choosing each response for each question in 
Section 1 is given in Table 12. The correct response (key) for each question 
is also identified. 


Table 12: Grade 6 Mathematics: Section 1 
Results for Individual Questions 


Distribution of 
Responses* (%) 


Distribution of 
Responses* (%) 


1 Cc 24 3.4 85.4 5.6 29 Cc -7 10.2 65.0 13.6 
2 D -5 23.1 6.6 66.6 30 B 9 59.4 8.8 2.6 
3 Cc .2 23.0 60.1 5.5 31 B 0 91.1 3.0 1.6 
4 c -5 12.0 59.6 8.5 32 A .2 24.9 9.1 24.2 
5 Cc -7 14.4 58.0 20.8 33 Cc .2 3.6 39.1 34.8 
6 Cc 1 3.4 73.0 7.3 34 B -6 61.1 17.1 18.7 
7 A al 7.5 2.1 31.1 35 B -5 66.8 14.1 11.1 
8 Cc 8 7.9 64.6 12.5 36 Cc 1 7.7 53.5 4.2 
9 D .5 8.9 15.0 67.2 37 D 3 3.8 6.9 85.4 
D -6 16.5 7.5 68.0 38 A 9 23.2 1.6 8.9 
D .7 7.9 22.6 59.7 39 B 5 91.3 2.7 4.0 
A 5 3.4 4.8 1.2 40 A 8 7.9 6.6 3.2 
D 2 1.4 6.2 85.2 41 B -5 69.5 2.4 26.1 
D -O 24.9 29.9 44.0 42 D 8 3.0 5.5 77.0 
B .7 84.4 7.4 3.3 43 A .0 5.4 9.9 23.1 
A 3 6.7 2.7 1.2 44 A .5 3.8 8.6 9.3 
D -l 4.4 7.7 85.5 45 A .7 5.9 11.7 12.9 
Cc as) 5.8 68.1 7.5 46 A -4 33.2 2.3 4.4 
D 2 3.5 7.3 86.9 47 B -3 64.5 5.9 25.3 
B -9 76.4 3.2 17.3 48 Cc -l 13.2 79.7 2.1 
D -2 5.7 6.9 76.1 49 B .8 91.8 2.7 2.6 
A -0 26.5 21.3 2.7 50 B -4 77.2 4.7 5.7 
Cc -6 13.3 63.2 11.9 51 A i3 9.0 8.1 17.2 
B 4° =75.3 6.3 5.8 52 Cc .0 6.7 81.5 2.4 
B -8 48.2 37.3 7.4 53 D .7 8.3 13.7 70.7 
D -l 29.7 7.0 54.5 54 A -2 20.7 18.6 3.6 
D me) 9.0 12.3 67.7 55 Cc 9 4.1 84.7 6.4 
Cc .2 3.3 85.3 3.9 


*The sum of the percentages for each question may be less than 100% because the 
No Response category is not included. This category is less than 2% for any 
one of these questions. 
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Section Il: Distribution of Jurisdiction Levels of Achievement 
Table 13 indicates the percentage of jurisdictions classified as significantly 
above, not different from, or below the provincial average for each reporting 
category in Section l. 
Table 13: Grade 6 Mathematics: Section 1 
Distribution of Jurisdiction Levels of Achievement 


Distribution of Jurisdictions 


Reporting Category Below Not Different Above 
Provincial from Provincial Provincial 
Average Average Average 
(%) (%) (%) 


Section 1 (total) 


Subject Matter 
Numeration 
Operations and Properties 
Measurement 
Geometry 
Graphing 


Problem-Solving Strategies 


Cognitive Level 
Knowledge 
Comprehension 
Application 


Problem-Solving Skills 
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Results for Section 2 -— Basic Facts 


This section of the test measures student mastery of basic facts; that is, it 
measures Student speed and accuracy. 


Results for Speed and Accuracy 
Table 14 presents the results in terms of the number of questions the students 
could answer correctly in 2 minutes, and in terms of the number of questions 


the students attempted in 2 minutes. 


Table 14: Grade 6 Mathematics: Section 2 
Speed and Accuracy for Basic Facts 


Average Number 


Number of Time of Questions Average Number 
Questions on Allotted Answered of Questions 
Operations the Test (min) Correctly . Attempted 

Addition 48 2 37.3 38.1 
Subtraction 48 2 35.2 36.6 
Multiplication 48 2 40.5 41.7 
Division 48 2 35.0 37.4 
Mixed Operations 48 2 36.4 38.7 


Frequency Distribution of Basic-Fact Scores 


Table 15 on page 20 presents total raw scores as well as their relative 
frequencies for each of the basic-fact tests. Any relative frequency smaller 
than 0.05 was rounded to 0.0. For example, 9 students or 0.03% obtained a 
score of 11 on addition, but the relative frequency for this score is recorded 
as 0.0. 
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Section 2 


Table 15: Grade 6 Mathematics: 
Relative Frequency Distribution of Basic-Fact Raw Scores 


Relative Frequency 
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GRADE 9 SOCIAL STUDIES ACHIEVEMENT TEST 


General Description 


The Grade 9 Social Studies Achievement Test is a two-part test. 
Part A: Multiple Choice consists of 60 questions, worth 70% of the total test 
score. 


Part B: Written Response is worth 30% of the total test score and requires 
the student to respond to short answer questions and to write a letter toa 
newspaper editor. All questions in this second part are linked to a current 
social issue. 


Content 

The test is based on the 1981 Alberta Social Studies Curriculum. All test 

questions are drawn from the content of the three topics prescribed for 
These topics are: Selected Market Economies, Selected Centrally 

Planned Economies, and Industrialization in Canada. Content emphasis is 


derived from the Grade 9 Social Studies Curriculum Specifications (Revised 
April 1986). 


FIO Se 


Value, knowledge, and skill objectives are measured in Part A and Part B of 
the test. Objectives related to the development of attitudes and some 
participation skills do not form part of this test. Although these objectives 
are important, questions based on such objectives yield different data than is 
ordinarily associated with student performance on an achievement test. 


“Part A: Multiple Choice 
The topics and objectives on which the multiple-choice questions of the test 


are based are shown in-the table below. 


Table 16: Grade 9 Social Studies - Number and Distribution of 
Multiple Choice Questions 


Value Knowledge Skill 
Content Area Objectives Objectives Objectives Total 
Topic A 8 9 20 
Topic B 9 8 20 
Topic C 9 20 
26 60 


The blueprint for Part A: Multiple Choice is presented in Table 17 on page 22. 
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TABLE 17: Grade 9 Social Studies - Test Blueprint 
Part A: Multiple Choice 


CONCEPT ELECTED MARKET ELECTED CENTRALLY INDUSTRIALIZATION 


REPORTING NCEPTS: ECONOMIES: PLANNED ECONOMIES: IN CANADA: 
CATEGORIES 


Competing values | Facts, concepts, Facts, concepts, and Facts, concepts, and 

and value and generalizations | generalizations. generalizations 

positions related to selected | related to selected related to industrial- 
PROCESS market economies centrally-planned ization in Canada 
REPORTING economies PROPORTION OF 
CATEGORIES TOTAL SCORE 


N NSION: , 
Recalls or recognizes data 12, 23, 36, 38, 24, 25, 26, 27, 29, : 48, 49, 50, 


and transforms data into 42, 55, 59 30, 31, 32, 33 54 
other words 


INQUIRY SKILLS I: 

Identifies an issue, -- 5, 16 28, 34, 35, 37 
formulates appropriate 

research questions, 

and gathers and organizes 

data 


INQUIRY SKILLS II: 
Analyses, evaluates, and 21, 22, 39, 40 
synthesizes data 


INQUIRY SKILLS IIT: 


Resolves issues, plans 18 
courses of action, and 
evaluates decisions and 
courses of action 


VALUING SKILLS: 


Analyses competing values 


PROPORTION OF TOTAL SCORE 


Part B: Written Response 


The written-response section of the test consists of 5 questions. Questions 
to 4 require short-answer responses. Question 5 asks the student to write a 
letter to a newspaper editor. The objectives on which the written-response 
questions are based are shown in Table 18. 
Table 18: Grade 9 Social Studies -— Test Blueprint 
Part B: Written Response 
PROPORTION 
DESCRIPTION OF OF TOTAL 
REPORTING CATEGORY WRITING ASSIGNMENT SCORE (%) 
I. Short Answer 1. Identifies speakers and value 5 
(Identification positions. Value objectives -—- 
of the Elements develop understanding of values and 
of an Issue) analyse values. 
2. Recalls facts related to an 4 


issue. Knowledge objectives -—- 
recall knowledge. 


3. Identifies possible courses of 4 
action. Skill objectives -- analyse 
and evaluate data. 


4. Formulates a generalization —- 2 
skill objectives -- synthesize 
data. 
Subtotal 15 
II. Letter 5. Present and defend a position. 15 
{Resolution of Skill objectives -- resolve the 
an Issue) issue and communicate effectively. 
Subtotal 15 
TOTAL 30 


="93 = 


1 


Absentees and Exemptions 


Table 19: Grade 9 Social Studies - Students Tested, 
Absentees, and Exemptions 


Number of 
Students Per Cent 


Students tested 27 731 
Students absent 1 113 
Students exempt 1 408 
Special needs 643 
Not taught subject this term 594 
ESL classes 123 
Language of Instruction 12 
not English 


Other (Approved by Director) 36 


Note: the data are incomplete as not all schools returned absentee and 
exemption reports. , 
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Qroee G — Sotek Sutrer 
RESULTS 


The report is based on the results for 27 731 Grade 9 students from public, 
separate, and private schools. 


Total Test 


The provincial average for the total test, which consisted of multiple-choice 
and written-response components, was 61.4%. The standard deviation* was 16.6. 


The total test score was obtained by combining the scores for Part A: Multiple 
Choice and Part B: Written Response so that the two parts had weightings of 
70% and 30% respectively. 


Standards for the Minimum Acceptable and Outstanding Levels of Achievement 


The scores representing minimum acceptable and outstanding levels of 
achievement were established by a standard setting procedure**, For the 
Grade 9 Social Studies Achievement Test the standards established were as 
follows: 


To meet the minimum acceptable performance level, students must achieve a 
raw score of 45. 


To meet the outstanding performance level, students must achieve a raw 
score of 83. 


Based on these standards, results were as follows: 


82.6% of the students performed at or above the minimum acceptable 
performance level. 


9.6% of the students performed at or above the outstanding performance 
level. 


The percentage of students who obtained each score on the test (relative 
frequency) and the percentage of students scoring at or below each score 
(cumulative frequency) are presented in Table 20 on page 26. 


*A measure of the variability of scores. Ina normal distribution, 68% of the 
students’ scores would fall within one standard deviation above and below the 
average. 


**An explanation of standard setting procedures is given in Appendix A. 
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Table 20: Grade 9 Social Studies - 
Frequency Distribution of Raw Scores 


Relative Cumulative Relative Cumulative Relative Cumulative 
Raw Frequency frequency Raw Frequency Frequency Raw Frequency Frequency 
Score (%) (%) Score (%) (%) Score (%) (%) 
0 34 0.8 6.5 68 2.1 62.3 
1 35 0.8 7.3 69 2.3 64.6 
2 36 0.9 8.2 70 2.1 66.7 
3 37 0.9 9.0 71 2.1 68.8 
4 38 ea 10.2 72 2.4 71.2 
5 39 ied 11.3 73 2.1 73.4 
6 40 1.2 12.5 74 2.1 75.5 
7 4) 1.2 13.7 75 2.2 77.6 
8 42 1.2 14.8 76 2.0 79.6 
9 43 1.3 16.1 77 2.0 81.6 
- 44 1.3 17.4 78 1.8 83.4 
0.0 45 13. 18.7 79 1.8 85.2 
- 46 1.4 20.1 80 1.8 87.0 
0.1 47 1.4 21.6 81 Ved 88.7 
0.1 48 1.5 23.1 82 1.6 90.4 
0.1 49 1.6 24.7 83 1.4 91.8 
- 0.2 50 1.7 26.4 84 1.3 93.1 
Os] 0.2 51 1.7 28.1 85 1.2 94.2 
0.1 0.3 52 lee 29.8 86 1.0 95.3 
0.1 0.4 53 1.8 31.7 87 0.9 96.1 
0.1 0.5 54 1.9 33:55 88 0.9 97.0 
0.2 0.7 55 1.9 35.5 89 0.8 97.7 
0.2 0.9 56 1.8 37.3 90 0.5 98.3 
0.2 Tea} 57 2.0 39.3 91 0.5 98.7 
0.3 1.4 58 2.1 41.3 92 0.4 99.1 
0.3 1.7 59 2.0 43.4 93 0.3 99.4 
0.3 2.0 60 2.0 45.4 94 0.2 99.7 
0.4 2.3 61 2.1 47.5 95 0.2 99.8 
0.4 rie | 62 2.2 49.7 96 0.1 99.9 
0.5 3.3 63 2.2 51.8 97 - 99.9 
0.6 3.8 64 2.1 53.9 98 - 100.0 
0.7 4.5 65 2.1 56.0 99 - 100.0 
0.6 5.1 66 2.0 58.0 100 - 100.0 
0.7 5.8 67 2.2 60.2 


Any score that was achieved by fewer than 0.05% of the population is 
represented by a dash (-). It should be noted, therefore, that the range of 
student scores was from 3 to 100, although the relative frequencies at the 
extreme ends of the distribution do not appear to indicate this. For example, 
one student achieved a raw score of 3 but, since this represents fewer than 
0.05% of the population, the relative frequency is shown by a dash (-). 
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Table 21 provides averages and standard deviations for the multiple-choice and 
written-response questions. Averages are in raw scores, which are the average 
number of marks obtained on each part of the test. 


Table 21: Grade 9 Social Studies - 
Multiple Choice and Written Response Raw Score Totals 


Total 
Test Possible Weighting Raw Score Standard 
Section Marks (%) Average Deviation 
Multiple Choice 60 70 38.5 10.3 
Written Response 30 30 16.6 5.9 


Table 22 contains the scores representing the minimum acceptable and 
outstanding levels of achievement on the multiple-choice and written-response 
portions of the test, as well as the percentages of students who achieved 
these standards. 


Table 22: Grade 3 Social Studies - 
Comparison of Performance to Standards: 
Multiple Choice and Written Response 


Minimum Acceptable Level Outstanding Level 
Raw Score Raw Score 
Representing | Students Meeting | Representing | Students Meeting 
Standard Standard (%) Standard Standard (%) 
Multiple 29 81.0 50 16.0 
Choice 
Written 12 80.7 25 8.4 


Response 


a ee 


Results for Part A: Multiple Choice 


The results for Part A are summarized by reporting category in Table 23. 
Although performance in the different reporting categories appears to show 
some variation, it is important to note that these scores are not directly 
comparable. The sets of questions that make up each reporting category were 
not selected to be equal in their average level of difficulty, therefore 
differences may be due to variations in question difficulty rather than in 
student performance. The averages can be used, however, in combination with 
jurisdiction results to detect patterns of relative strength or weakness in 
achievement in each of the reporting categories. 


Table 23: Grade 9 Social Studies - Results by Reporting Category 
Part A: Multiple Choice 


No. of Raw Score Standard 
Reporting Category Questions Average Deviation 


Total Score. (Multiple Choice) 
Topic A: Selected Market Economies 


Topic B: Selected Centrally- 
Planned Economies 


Topic C: Industrialization in Canada 


Recall and Comprehension 
All Topics 
Topic A 
Topic B 
Topic C 


Value Concepts and Valuing Skills 
(all topics) 


Inquiry Skills I (all topics) 


Inquiry Skills II (all topics) 


Inquiry Skills III (all topics) 


= 99% 


Table 24 shows the frequency distribution of raw scores on Part A of the test. 


Table 24: Grade 9 Social Studies - Frequency Distribution of Raw Scores 
Part A: Multiple Choice 


Relative Cumulative Relative Cumulative Relative Cumulative 
Raw Frequency Frequency Frequency Frequency Raw Frequency Frequency 
Score (%) (%) (%) (%) (%) (%) 
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1.0 
1.3 
1.4 
1.5 
1.5 
1.9 
2.0 
2.2 
1.9 
2.2 
2.5 
2.6 
2.7 
2.8 
2.9 
2.8 
3.0 
3.2 
3.2 
3.2 
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*Relative Frequency: the percentage of students achieving each score. 
**Cumulative Frequency: the percentage of students achieving at or below each 
score. 


Any score that was achieved by fewer than 0.05% of the population is 
represented by a dash (-). It should be noted, therefore, that the range of 
scores was from 1 to 60, although the relative frequencies at the upper and 
lower ends of the distribution do not appear to indicate this. For example, 
five (5) students achieved a raw score of 60 but, since this represents fewer 
than 0.05% of the population, the relative frequency is shown by a dash (-). 
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The response frequencies for all 60 multiple-choice questions appearing in the 


test are presented in Table 25. 


Grade 9 Social Studies - Response Frequencies 


Table 25: 


Multiple Choice 


Part A: 


Distribution of 
Responses (%)* 


Distribution of 
Responses (%)* 


19.3 


9.0 12.4 


4.8 69.4 


78 


72.3 


6.0 14.9 
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19.1 
2 


7.6 
8.8 
3.8 
4.2 


62.4 10.5 


9.1 


17.4 15.5 57.8 


13.3 


16.4 56.1 


14.1 


41.4 21.2 12.9 24.3 


43.8 13.7 45 


5.5 36.8 


14.9 
7.4 59.5 


46 


-8 


12 


47 


48 


70.9 


5.5 


9 


12 


49 


50 
51 


52 


53 


11.4 59.5 


55 


7.5 


9.2 44.7 


56 


9.2 


10.1 26.2 54.4 


5.2. 4.6 83.7 


7.7 


3 


58 


22.0 59.8 


59 


60 


4.0 


7.9 83.3 


*The sum of the percentages may be less than 100% because the No Response 


category is not included and the numbers have been rounded. 
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Results for Part B: Written Response 


The results for the written-response questions are summarized, by question, in 


Table 26. 
Table 26: Grade 9 Social Studies - 
Average Scores Awarded for Written-Response Questions 
Total Marks 
Question Possible Average Score Difficulty Level* 


Short Answer 


0.70 
0.49 
0.55 
0.40 


0.54 


*The difficulty level is the average score divided by total marks possible. 


! 


2.37 


The distribution of scores for the short-answer written-response questions 
(questions 1 to 4) are presented in Table 27. 


Table 27: Grade 9 Social Studies - Distribution of Scores 
for Short-Answer Written-Response Questions 


Percentage of Students Obtaining Each Mark 


Score 
Question NR* 0 1 2 3 4 5 
1 1.2 2.9 8.9 9.8 19.7 23.5 34.0 
2 5.8 15.1 19.4 22.8 17.9 18.9 - 
3 5.0 8.0 16.9 28.4 21.2 20.5 - 
4 12.2 36.0 22.5 29.1 - - ~ 


*No Response 


The distribution of scores for the letter (question 5) are presented in 
Table 28. 


Table 28: Grade 9 Social Studies - 
Distribution of Scores on Question 5 


Distribution of 


Scale Points Responses (%) 
5 (Excellent) 4.8 
4 (Good) 16.0 
3 (Satisfactory) 38.1 
2 (Limited) 30.6 
1 (Poor) 6.8 
0 (Off topic, insufficient response, 3.6* 


or blank paper) 


*2.6% of the total papers were blank on Question 5. 


2.899 


-_ 
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Distribution of Jurisdiction Levels of Achievement ( * j 


Table 29 indicates the percentage of jurisdictions classified as significantly 
above, not different from, or below the provincial average for each subtest. 


Table 29: Grade 9 Social Studies - Distribution of Jurisdiction 
Levels of Achievement — 


Distribution of Jurisdictions (%) 


Reporting Category Below Not Different Above 
Provincial from Provincial Provincial 
Average Average Average 

Total Test 24.0 54.9 21.1 

Multiple Choice 

Total Score 25.5 53.9 20.6 

Topic A: Selected Market 24.6 59.1 16.3 
Economies 

Topic B: Selected Centrally- 24.5 60.3 15.2 
Planned Economies 

Topic C: Industrialization in 21.3 60.4 18.3 
Canada 

Recall and Comprehension: 
All Topics 29.1 52.2 18.7 
Topic A 32.0 57.1 10.8 
Topic B 26.4 55.2 18.4 
Topic C 21.7 59.1 19.2 

Value Concepts and Valuing Skills 22.1 65.2 12.7 
(all topics) 

Inquiry Skills I (all topics) 17.8 67.8 14.4 

Inquiry Skills II (all topics) 12.4 65.3 22.3 

Inquiry Skills III (all topics) 16.0 70.0 14.0 

Written Response 

Total Score 22.5 57.8 19.6 

Short Answer Questions 20.6 62.7 16.7 

Letter 23.0 69.6 7.4 


GUIDE TO THE INTERPRETATION OF JURISDICTION RESULTS , 
Achicne wou Tesrs ~ i498] - ere ODL WAV refer , 


In addition to their use in monitoring student achievement for the province as 
a whole, the results of the achievement tests are useful in comparing 
achievement in a particular jurisdiction with provincial results. However, 
care must be exercised in making these comparisons and in drawing conclusions 
from the data. 


The following jurisdiction and school reports for each subject are provided 
for each jurisdiction under separate covers. 


1. The Jurisdiction Summary Report contains jurisdiction equivalents of 
the provincial results that are given in all statistical tables in 
the report. 


2. The School Summary Report contains the school equivalents of the 
provincial results that are given in all statistical tables in this 
report. 


3. Individual Student Subtest Results are reported for each school. 


These reports are confidential to the jurisdiction. 


Differences Between Jurisdiction and Provincial Averages 


Jurisdictions are provided with their average scores for each reporting 
category. These scores may be compared to the provincial average for the same 
reporting category. However, the importance of differences between 
jurisdiction averages and provincial averages is not always clear. To aid in 
the interpretation of differences between the averages, jurisdiction and 
school reports indicate when the difference is unlikely to be due to chance 
variation in the abilities of students. For the purposes of the provincial 
testing program, the 95% confidence interval is used. That is, if the 
probability is less than 1 in 20 that the difference is due to chance, the 
difference is very likely a real difference, and the jurisdiction average is 
classified as different from the provincial average. Otherwise, it is 
classified as not different from the provincial average. The provincial 
average for that reporting category determines the true population average. 
The standard deviation for the jurisdiction is used to estimate the standard 
error of the mean. 


Because achievement levels are calculated by taking jurisdiction size into 
consideration, two jurisdictions with the same averages but of different sizes 
may be classified differently. The larger jurisdiction would be more likely 
to be above or below average, because the amount of chance variation would be 
less in larger jurisdictions, and the actual difference would represent a 
larger variation from the provincial average. 
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A test score does not indicate why a particular performance occurred, only 
that it did occur. Identification of reasons for that performance should be 
undertaken only when the results have been studied. There are a variety of 
factors that should be examined: 


1. 


Student motivation. Consideration should be given to the degree to 
which students were motivated to perform to their levels of ability. 


Student ability. While the notion of confidence interval is designed 
to take into consideration year-to-year fluctuations in the average 
ability levels of students, it is possible that a group of students 
with a particularly high or low average ability may come through a 
system. This is much more likely to be a factor in small systems 
than in large ones. 


Teaching and curriculum. Consideration should be given to the type 


of instruction students have received in the jurisdiction and the 
adequacy of curriculum implementation. 
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APPENDIX A: STANDARD SETTING PROCEDURE 


Provincial averages are useful for comparing the performance of students ina 
school or jurisdiction to overall levels of achievement. However, if used 
alone, provincial averages do not indicate whether achievement levels are what 
they should be. A test score by itself has limited meaning without comparison 
to a standard. A raw score of 25/50, for example, could represent very high 
achievement on one test, and very low achievement on another. 


Grade 3 Science and Grade 6 Mathematics 


For the Grade 3 Science and the Grade 6 Mathematics, identical procedures were 
used to establish standards that allow for the assessment of achievement on 
the respective tests. The Student Evaluation and Records Branch asked 30 
experienced Grade 3 Science and Grade 6 Mathematics teachers respectively to 
examine each question on the test and determine the expected difficulty level 
of that question for two groups of students: borderline passing students and 
borderline outstanding students. From the individual question difficulty 
levels, the overall test difficulty levels expected for these two groups of 
students were determined. The averages of the test difficulty levels 
established by the teachers provided the standards for minimum acceptable 
performance and outstanding performance on the test. 


Grade 9 Social Studies 


For the Grade 9 Social Studies test, experienced Grade 9 teachers from all 
parts of the province met to determine what raw score would be expected on the 
test for two categories of borderline student. The minimum acceptable 
borderline is the division between the student who could be expected to 
minimally achieve the objectives, and one who could not. The outstanding 
borderline divides students who could be expected to achieve an outstanding 
level of performance and those who could not. 


After a review of the prescribed curriculum, it was judged that, given 
adequate teaching and resources, 85% of the Grade 9 students should be able to 
Minimally achieve the objectives of the Grade 9 Social Studies curriculum, as 
reflected by the achievement test. Since 85% of the students should be able 
to reach this level, the borderline student would be at the 15th percentile in 
ability. The outstanding borderline student would be at the 85th percentile 
since it was judged that 15% of the students should be able to reach this 
level given adequate teaching and resources. 


The teachers examined each question on the test and determined the difficulty 
of that question for a 15th percentile student. (Question 5 on the written-— 
response section of the test was excluded from this process, as an implicit 
standard is built into the scoring guide.) From difficulty levels of 
individual questions, the overall test difficulty for the minimally acceptable 
borderline student was determined. The average of the difficulty levels 
established by the teachers is the standard for the test. For the borderline 
outstanding student, global judgments about the difficulty of the total test 
and its two major subsections were made following the previous item-by-item 
review of the test. 
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